-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stratum v2 Template Provider via IPC interface (multiprocess) #48
base: 2024/07/sv2-tp-common
Are you sure you want to change the base?
Conversation
@ryanofsky I slightly mangled e373642 because of bitcoin@4e1a434#r1681193702. I'll clean that up later. |
Incorporated the changes from bitcoin#30437 (comment) and cleaned up commit history. |
@ryanofsky do you have a commit somewhere that adds libmultiprocess to the guix build? |
c75792b
to
91720b7
Compare
Getting a cryptic Somehow related to the Template Provider depending on I can't make sense of the |
db58076
to
1ff4660
Compare
4354e96
to
3aa6f0a
Compare
1ff4660
to
ee010f5
Compare
re: #48 (comment)
I don't, since I haven't really experimented with the guix build. Maybe simplest approach to start with would enable the multiprocess build option and build and package all the multiprocess executables including bitcoin-tp and bitcoin-node. I know in the longer run you want to have separate releases as described in bitcoin#30437 (comment). re: #48 (comment)
This is a -bitcoin_mine_CPPFLAGS = $(bitcoin_bin_cppflags)
+bitcoin_mine_CPPFLAGS = $(bitcoin_bin_cppflags) $(BOOST_CPPFLAGS) to add the right -isystem option.
Not sure about this error. Link seems to be https://cirrus-ci.com/task/4759864851824640?logs=ci#L2486 |
19cc9f1
to
dabc32c
Compare
6c24b45
to
a8bb152
Compare
I'm getting the MSAN / TSAN error inconsistently on my self-hosted CI. Does not seem related to this PR. Trying to reproduce in isolation in #51. |
When ENABLE_IPC option is on, build with libmultiprocess subtree and `add_subdirectory(src/ipc/libmultiprocess)` instead of external package and `find_package(Libmultiprocess)` by default. Behavior can be toggled with `WITH_LIBMULTIPROCESS` option. Using a subtree should be more convenient for most bitcoin developers, but using an external package is more convenient for developing in the libmultiprocess repository. The `WITH_LIBMULTIPROCESS` option is also used to avoid needing to changing the depends build here. But in later commits, the depends build is switched to use the add_subdirectory build as well. Co-authored-by: Cory Fields <[email protected]>
This change is technically not needed to add libmultiprocess as a subtree, but it avoids a CI failure in followup PR bitcoin#30975 which enables multiprocess build option in more CI jobs. In that PR, several jobs fail due to the mptest executable not being built by default, as reported bitcoin#30975 (comment)
This change is technically not needed to add libmultiprocess as a subtree, but it avoids a CI failure in followup PR bitcoin#30975 which enables multiprocess build option in more CI jobs. In that PR, the "macOS 14 native no depends job" fails due to warnings in boost headers treated as errors, reported in bitcoin#30975 (comment) and chaincodelabs/libmultiprocess#138
This change is technically not needed to add libmultiprocess as a subtree, but it avoids a CI failure in followup PR bitcoin#30975 which enables multiprocess build option in more CI jobs. In that PR, clang-tidy job fails due to missing generated example files as reported bitcoin#30975 (comment) Different fixes were suggested bitcoin#30975 (comment) and bitcoin#30975 (comment) Co-authored-by: Hennadii Stepanov <[email protected]>
Co-authored-by: Cory Fields <[email protected]>
Without this change linter produces errors about: - Use of std::filesystem the libmultiprocess example program. - Use of locale-dependent functions in example program, in the build time code generator, and in the runtime library for debug logging. - Include guards not beginning with BITCOIN_
Move parts of the int_get_build_id into a new int_get_build_properties function. There is no change in behavior. This just organizes assignments better so some build properties can be used to help compute build ids in the next commit.
With newly introduced libmultiprocess subtree, there's no need for depends system to download and track changes to the upstream repository. Note that adding the libmultiprocess subtree does not allow dropping libmultiprocess packages from the depends build, because libmultiprocess includes a code generation tool called mpgen, and in cross-compiled builds, bitcoin core's cmake build system doesn't have access to a native toolchain and can't build mpgen itself, so the depends system (or the native environment if not using depends) needs to supply it.
Co-authored-by: Hennadii Stepanov <[email protected]>
Except: 1. i686, DEBUG (changed to "no multiprocess") 2. Windows due to lack of support
Install capnp where needed. The bitcoin-node binary is built on all platforms which have multiprocess enabled, but for functional tests it's only used in the macOS native (no depends) and CentOS native (depends) jobs.
Except for Windows and OpenBSD.
Allows InitError and InitWarning to be used by other executables beside bitcoind and bitcoin-node.
See src/bitcoin-mine.cpp for usage information. Co-authored-by: Sjors Provoost <[email protected]>
For now also build bitcoin-node.
The Template Provider binary: * won't have a wallet * probably won't have a gui * doesn't use the functional test framework Disable some CI stuff to speed it up. This is not a complete list, because: 1. That would be too hard to rebase 2. Some things might be added later, e.g. tracepoints
This is an alternative approach to #68. Rather than integrating the Template Provider directly into Bitcoin Core, we make it an entirely separate application that communicates via IPC, see design/multiprocess.md.
Usage
Compile as usual, multiprocess is enabled by default.
Alternatively get a recent binary, see (unofficial, by me) releases ending in "ICP".
Start node:
Start the Template Provider:
See general Stratum v2 usage instructions in bitcoin#29432.
Rationale, PRs this builds on
There are multiple ways to create such an application. It could be built from scratch in Rust, using bits and pieces of the Stratum Reference Implementation SRI. It could be a c++ application built from scratch, perhaps using some libraries from Bitcoin Core.
But this PR goes about in a slightly different way. Similar to Elements or Knots it's a set of patches on top of the full Bitcoin Core codebase. This allows me to leverage the build system, subclass
Transport
and use various other useful bits.This lets me reuse almost all code from the integrated Template Provider bitcoin#29432. Both build direct on:
This PR relies on the interface and build changes listed in bitcoin#31098. These interface changes are also used by bitcoin#29432, but there they run in the same process.
I modified the guix script to only produce
bitcoin-node
(for now) andbitcoin-mine
binaries.Goal
The goal is to drop
bitcoin-node
from the release here. Ideally users would install Bitcoin Core in the manner they're familiar with. They would then installbitcoin-mine
separately and it should just work(tm), at most having to add a line tobitcoin.conf
to turn IPC on.This won't work until we ship a release with multiprocess and the mining interface enabled, ideally in
bitcoind
andbitcoin-qt
rather than in seperatebitcoin-node
andbitcoin-gui
binaries. See bitcoin#30437 for progress towards that.Rebase hint
Rebase hint: